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Abstract: These lecture notes are an informal introduction to the theory of 
computational complexity and its links to quantum computing and statistical me- 
chanics. 



1 Introduction 

1.1 Motivation 

Compared to the traditionally close relationship between physics and mathematics, an ex- 
change of ideas and methods between physics and computer science barely takes places. The 
few interactions that go beyond Fortran programming and the quest for faster computers were 
often successful and provided surprising insights in both fields. This holds particularly for the 
mutual exchange between statistical mechanics and the theory of computational complexity. 

The branch of theoretical computer science known as computational complexity is con- 
cerned with classifying problems according to the computational resources (CPU time, mem- 
ory) required to solve them. This has lead to precisely defined notions of tractable and in- 
tractable problems. It appears as if this notions can be transferred to the problem of analytical 
solubility of models from statistical physics, explaining to some extent why for example the 
Ising model is exactly soluble in two dimensions, but not in three (sec. |3T2| ) . 

The success of the theory of computational complexity is based on a pessimistic attitude: 
a problem's tractability is defined according to the worst possible instance. Quite often this 
worst case scenario differs considerably from the typical case, averaged over a reasonable 
ensemble of instances. A common observation is that hard problems are typically easy to 
solve. To get real hard instances, the parameters of the underlying ensemble must be carefully 
tuned to certain critical values. Varying the parameters across the critcal region leads to abrupt 
changes in the typical complexity — very similar to the abrupt changes associated with phase 

published in Computing in Science & Engineering 4 31^47 (May/June 2002) 
Jtephan.mertens@physik.uni-magdeburg.de 



1 



transitions in physical systems. Phase transitions in physical and computational systems are 
best studied within the framework of statistical mechanics (sec. 

Apart from phase transitions, statistical mechanics offer means for the general probabilistic 
analysis of computational problems. The usual way is to formulate an optimization problem as 
a spin glass and analyze the low temperature properties of the latter. This 'physical' approach 
often yield results which go beyond the results obtained by traditional methods (sec. |33| ). 

Another exciting link between physics and computer science is provided by quantum com- 
puting. There is some theoretical evidence that computers using quantum systems as computa- 
tional devices are more powerfull than computers based on classical devices. The hope is that 
problems which are intractable on a classical computer become tractable when put on a quan- 
tum computer. Results like Shor's celebrated quantum algorithm for factorization nurture this 
hope, but a real breakthrough is still missing. Obviously some knowledge of computational 
complexity helps to understand the promises and limitations of quantum computers (sec. |3T| ). 

These notes are directed at physicists with no or little knowledge of computational com- 
plexity. Of course this is not the first informal introduction into the field, see e.g. [[[], 0] or 
the corresponding chapters in textbooks on algorithms For a deeper understanding of 

the field you are referred to the classical textbooks of Garey and Johnson [j|] and Papadim- 
itriou [§]. 

1.2 The measure of complexity 

1.2.1 Algorithms 

The computational complexity of a problem is a measure of the computational resources, typi- 
cally time, required to solve the problem. What can be measured (or computed) is the time that 
a particular algorithm uses to solve the problem. This time in turn depends on the implemen- 
tation of the algorithm as well as on the computer the program is running on. The theory of 
computational complexity provides us with a notion of complexity that is largely independent 
of implementational details and the computer at hand. Its precise definition requires a con- 
siderable formalism, however. This is not surprising, since it is related to a highly non trivial 
question that touches the fundaments of mathematics: What do we mean by saying a problem 
is solvable? Thinking about this question leads to Godel's incompleteness theorem, Turing 
machines and the Church Turing Thesis on computable functions. See [0] for an entertaining 
introduction into these topics written by a physicist. 

Here we will adopt a more informal, pragmatic point of view. A problem is solvable if it 
can be solved by a computer program written in your favourite programming language. The 
running time or time complexity of your program must then be defined with some care to serve 
as a meaningful measure for the complexity of the problem. 

1.2.2 Time complexity 

In general the running time depends on the size of the problem and on the specific input 
data, the instance. Sorting 1000 numbers takes longer than sorting 10 numbers. Some sorting 
algorithms run faster if the input data is partially sorted already. To minimize the dependency 
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on the specific instance we consider the worst case time complexity T(n), 

T(n) = maxt(x) (1) 

\x\=n 

where t(x) is the running time of the algorithm for input data x (in arbitrary units) and the 
maximum is taken over all problem instances of size n. The worst case time is an upper bound 
for the observable running time. 

A measure of time complexity should be based on a unit of time that is independent of 
the clock rate of a specific CPU. Such a unit is provided by the time it takes to perform 
an elementary operation like the addition of two integer numbers. Measuring the time in 
this unit means counting the number of elementary operations executed by your algorithm. 
This number in turn depends strongly on the implementation details of the algorithm - smart 
programmers and optimizing compilers will try to reduce it. Therefore we will not consider 
the precise number T{n) of elementary operations but only the asymptotic behavior of T(n) 
for large values of n as denoted by the Landau symbols O and 6: 

• We say T(n) is of order at most g(n) and write T(n) = 0(g(n)) if there exist positive 
constants c and n such that T{n) < cg(n) for all n > n . 

• We say T(n) is of order g(n) and write T{n) = Q(g(n)) if there exist positive constants 
ci, c 2 and n such that c\g(n) < T(n) < c 2 <?(n) for all n > n . 

Multiplying two n x n matrixes requires n 3 multiplications according to the textbook for- 
mula. Does this mean that the problem of multiplying two n x n matrices has complexity 
0(n 3 )? No. The textbook formula is a particular algorithm, and the time complexity of an 
algorithm is only an upper bound for the inherent complexity of a problem. In fact faster 
matrix multiplication algorithms with complexity 0(n a ) and a < 3 have been found during 
the last decades, the current record being a = 2.376 [§]. Since the product matrix has n 2 
entries, a can not be smaller than 2, and it is an open question whether this lower bound can 
be achieved by an algorithm. A problem where the upper bound from algorithmic complexity 
meets an inherent lower bound is sorting n items. Under the general assumption that com- 
parisons between pairs of items are the only source of information about the items, it can be 
shown that 0(n logn) is a lower bound for the number of comparisons to sort n items in the 
worst case [|3|, chap. 9]. This bound is met by algorithms like heapsort or mergesort. 

1.2.3 Problem size 

Our measure of time complexity still depends on the somewhat ambiguous notion of problem 
size. In the matrix multiplication example we tacitly took the number n of rows of one input 
matrix as the "natural" measure of size. Using the number of elements m = n 2 instead will 
"speed up" the 0(n 3 ) algorithm to 0(m 3 ^ 2 ) without changing a single line of program code. 
You see that an unambiguous definition of problem size is required to compare algorithms. 

In computational complexity, all problems which can be solved by a polynomial algo- 
rithm, i.e. an algorithm with time complexity 0(n fe ) for some k, are lumped together and 
called tractable. Problems which can only be solved by algorithms with non-polynomial run- 
ning time like 0(2 n ) or 0(n!) are also lumped together and called intractable. There are 
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practical as well as theoretical reasons for this rather coarse classification. One of the theoreti- 
cal advantages is that it does not distinguish between the 0(n 3 )- and 0(n 3 / 2 )-algorithm form 
above, hence we can afford some sloppiness and stick with our ambigous "natural" measure 
of problem size. 

1.3 Tractable and intractable problems 

1.3.1 Polynomial vs. exponential growth 

From a practical point of view, an exponential algorithm means a hard limit for the accessible 
problem size. Suppose that with your current equipment and you can solve a problem of size n 
just within the schedule. If you algorithm has complexity 6(2 n ), a problem of size n + 1 will 
need twice the time, bringing you definitely out of schedule. The increase in time caused by an 
0(n) or Q(n 2 ) algorithm on the other hand is far less dramatic and can easily be compensated 
by upgrading your hardware. You might object that a 6(n 100 ) algorithm outperforms a 0(2 n ) 
algorithm only for problem sizes that will never occur in your application. A polynomial al- 
gorithm for a problem usually goes hand in hand with a mathematical insight into the problem 
which enables you to find a polynomial algorithm with small degree, typically Q(n k ), k = 1, 2 
or 3. Polynomial algorithms with k > 10 are rare and arise in rather esoteric problems. 

1 .3.2 Tractable trees 

As an example consider the following problem from network design. You have a business 
with several offices and you want to lease phone lines to connect them up with each other. The 
phone company charges different amounts of money to connect different pairs of cities, and 
your task is to select a set of lines that connects all your offices with a minimum total cost. 




1 2 

Figure 1: A weighted graph and its minimum spanning tree (colored edges). 

In mathematical terms the cities and the lines between them form the vertices V and edges 
E of a weigthed graph G = (V,E), the weight of an edge being the leasing costs of the 
corresponding phone line. Your task is to find a subgraph that connects all vertices in the 
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graph, i.e. a spanning subgraph, and whose edges have minimum total weight. Your subgraph 
should not contain cycles, since you can always remove an edge from a cycle keeping all nodes 
connected and reducing the cost. A graph without cycles is a tree, so what you are looking for 
is a minimum spanning tree in a weighted graph (Fig. [j]). 

Minimum Spanning Tree (MST): Given a weighted graph G = (V, E) with 
non-negative weights. Find a spanning Tree T C G with minimum total weight. 

How to find a minimum spanning tree? A naive approach is to generate all possible trees with 
n vertices and keep the one with minimal weight. The enumeration of all trees can be done 
using Priifer codes [BR, but Cayley's formula tells us that there are n n ~ 2 different trees with 
n vertices. Already for n = 100 there are more trees than atoms in the observable universe! 
Hence exhaustive enumeration is prohibitive for all but the smallest trees. The mathematical 
insight that turns MST into a tractable problem is this: 

Lemma: Let U E V be any subset of the vertices of G = (V, E), and let e be the 
edge with the smallest weight of all edges connecting U and V — U. Then e is 
part of the minimum spanning tree. 

Proof: By contradiction. Suppose T is a minimum spanning tree not containing e. Let e = 
(u, v) with u E U and v E V — U. Then because T is a spanning tree it contains a unique path 
from u to v, which together with e forms a cycle in G. This path has to include another edge 
/ connecting U and V — U. Now T + e — / is another spanning tree with less total weight 
than T. So T was not a minimum spanning tree. 

The lemma allows to grow a minimum spanning tree edge by edge, using Prim's algorithm 
for example: 

Prim(G) 

Input: weighted graph G(V,E) 
Output: minimum spanning tree TEG 
begin 

Let T be a single vertex v from G 
while T has less than n vertices 

find the minimum edge connecting T to G — T 

add it to T 
end 
end 



The precise time complexity of Prim's algorithm depends on the data structure used to 
organize the edges, but in any case 0(n 2 logn) is an upper bound, (see [ jlliy for faster algo- 
rithms). Equipped with such a polynomial algorithm you can find minimum spanning trees 
with thousands of nodes within seconds on a personal computer. Compare this to exhaustive 
search! According to our definition, MST is a tractable problem. 
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1.3.3 Intractable itineraries 

Encouraged by the efficient algorithm for MST we will now investigate a similar problem. 
Your task is to plan an itinerary for a travelling salesman who must visit n cities. You are 
given a map with all cities and the distances between them. In what order should the salesman 
visit the cities to minimize the total distance he has to travel? You number the cities arbitrarely 
and write down the distance matrix (dy), where G? y - denotes the distance between city number 
i and city number j. A tour is given by a cyclic permutation n : [1 . . . n] \—> [1 . . . n], where 
7r(z) denotes the successor of city i, and your problem can be defined as: 

Traveling Salesman Problem (TSP): Given an x n distance matrix with 
elements > 0. Find a cyclic permutation n : [1 . . . n] i— > [1 . . . n] that mini- 
mizes 

n 

C n{n) = ^ diirjj) (2) 
i=l 

The TSP probably is the most famous optimization problem, and there exists a vast literature 
specially devoted to it, see [Jl~T] — [T4|] and references therein. It is not very difficult to find good 
solutions, even to large problems, but how can we find the best solution for a given instance? 
There are (n — 1)! cyclic permutations, calculating the length of a single tour can be done 
in time 0(n), hence exhaustive search has complexity 0(n\). Again this approach is limited 
to very small instances. Is there a mathematical insight that provides us with a shortcut to 
the optimum solution, like for MST? Nobody knows! Despite the efforts of many brilliant 
people, no polynomial algorithm for the TSP has been found. There are some smart and 
efficient (i.e. polynomial) algorithms that find good solutions but do not guarantee to yield the 
optimum [[12]]. According to our definition, the TSP is intractable. 




Figure 2: Same instance, different problems: A valid configuration of the traveling sales- 
man problem (left) and the ASSIGNMENT problem (right). Whereas ASSIGN- 
MENT can be solved in polynomial time, the TSP is intractable. 

Why is the TSP intractable? Again, nobody knows. There is no proof that excludes the 
existence of a polynomial algorithm for TSP, so maybe someday someone will come up with 
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a polynomial algorithm and the corresponding mathematical insight. This is very unlikely, 
however, as we will see soon. 

The intractability of the TSP astonishes all the more considering the tractability of a very 
similar, almost identical problem: 

Assignment: Given an x n cost matrix with elements > 0. Find a permu- 
tation 7T : [1 . . . n] i— > [1 . . . n] that minimizes 

Tl 

C «0) = 5^^7r(i) (3) 

i=l 

The only difference between TSP and Assignment is that the latter allows all permutations 
on n items, not only the cyclic ones. If the dij denote distances between cities, Assign- 
ment corresponds to total tour length minimization for a variable number of salesmen, each 
travelling his own subtour (Fig. |2|). 

The classical application of ASSIGNMENT is the assignment of n tasks to n workers, sub- 
ject to the constraint that each worker is assigned exactly one task. Let d^ denote the cost of 
having task j performed by worker i, and denote the task assigned to worker i, ASSIGN- 
MENT is the problem of minimizing the total cost. 

There are n\ possible assignments of n tasks to n workers, hence exhaustive enumeration 
again is prohibitive. In contrast to the TSP, however, Assignment can be solved in poly- 
nomial time, for example using the 0(n 3 ) hungarian algorithm [Q]. Compared to MST, the 
algorithm and the underlying mathematical insight are a bit more involved and will not be 
discussed here. 

2 Complexity classes 

2.1 Decision problems 

So far we have discussed optimization problems: solving MST, TSP or Assignment implies 
that we compare an exponential number of feasible solutions with each other and pick the 
optimum. Exhaustive search does this explicitely, polynomial shortcuts implicitely. Maybe we 
learn more about the barrier that separates tractable from intractable problems if we investigate 
simpler problems, problems where the solutions are recognizable without explicit or implicit 
comparison to all feasible solutions. So let us consider decision problems, problems whose 
solution is either 'yes' or 'no'. 

Any optimization problem can be turned into a decision problem by adding a bound B to 
the instance. Examples: 

MST (Decision): Given a weighted graph G = (V,E) with non-negative 
weights and a number B > 0. Does G contain a spanning tree T with total 
weight < Bl 

TSP (Decision): Given a n x n distance matrix with elements d^ > and a 
number B > 0. Is there a tour n with length < Bl 
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In a decision problem, the feasible solutions are not evaluated relative to each other but to an 
'absolut' criterion: a tour in the TSP either has length < B or not. 

MST(D) can be solved in polynomial time: simply solve the optimization variant MST 
and compare the result to the parameter B. For the TSP(D) this approach does not help. 
In fact we will see in section |2.3.1| that there exists a polynomial algorithm for TSP(D) if 
and only if there exists a polynomial algorithm for TSP. It seems as if we cannot learn more 
about the gap between tractable and intractable problems by considering decision variants of 
optimization problems. So lets look at other decision problems, not derived from optimization 
problems. 

2.1.1 Eulerian circuits 

Our first 'genuine' decision problem dates back into the 18th-century, where in the city of 
Konigsberg (now Kaliningrad) seven bridges crossed the river Pregel and its two arms (Fig. ^]). 
A popular puzzle of the time asked if it was possible to walk through the city crossing each of 
the bridges exactly once and returning home. 




Figure 3: The seven bridges of Konigsberg, as drawn in Euler's paper from 1736 [ |T5| ] (left) and 
represented as a graph (right). In the graph, the riverbanks and islands are condensed 
to points (vertices), and each of the bridges is drawn as a line (edge). 



It was Leonhard Euler who solved this puzzle in 1736 Q15|]. First of all Euler recognizes 
that for the solution of the problem the only thing that matters is the pattern of interconnections 
of the banks and islands-a graph G = (V, E) in modern terms. The graph corresponding to 
the puzzle of the Konigsberg bridges has 4 vertices for the two banks and the two islands and 
7 edges for the bridges (Fig. 0). Euler's paper on the Konigsberg bridges can be regarded as 
the birth of graph theory. 

To generalize the Konigsberg bridges problem we need some terminology from graph 
theory A walk in a graph G = (V, E) is an alternating sequence of vertices v E V and 
edges (v, v') e E, 

vx, (vi, v 2 ),v 2 , (t> 2 , v 3 ),..., (vi-X,Vi),Vi. 
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Note that the sequence begins and ends with a vertex, and each edge is incident with the 
vertices immediately preceding and succeeding it. A walk is termed closed if Vi — v\, open 
otherwise. A walk is called a trail if all its edges are distinct, and a closed trail is called a 
circuit. What the strollers in Konigsberg tried to find was a circuit that contains all edges. To 
the honour of Leonhard Euler such a circuit is called Eulerian circuit. Equipped with all this 
terminology we are ready to define the generalization of the Konigsberg bridges problem: 

Eulerian Circuit: Given a graph G = (V,E). Does G contain an Eulerian 
circuit? 

Obviously this is a decision problem: the answer is either 'yes' or 'no', and any circuit can be 
checked to be Eulerian or not without resorting to all possible circuits. 

Once again exhaustive search would solve this problem, but the intractability of this ap- 
proach was already noticed by Euler. More than 200 years before the advent of computers he 
wrote "The particular problem of the seven bridges of Konigsberg could be solved by carefully 
tabulating all possible paths, thereby ascertaining by inspection which of them, if any, met the 
requirement. This method of solution, however, is too tedious and too difficult because of the 
large number of possible combinations, and in other problems where many more bridges are 
involved it could not be used at all." (cited from [JT[]). 

Euler solved the Konigsberg bridges problem not by listing all possible trails but by math- 
ematical insight. He noticed that in a circuit you must leave each vertex via an edge different 
from the edge that has taken you there. In other words, the degree of the vertex (that is the 
number of edges adjacent to the vertex) must be even. This is obviously a necessary condition, 
but Euler proved that it is also sufficient: 

Theorem: A connected graph G = (V, E) contains an Eulerian circuit if and only 
if the degree of every vertex v E V is even. 

Euler's theorem allows us to devise a polynomial algorithm for Eulerian Circuit: Check 
the degree of every vertex in the graph. If one vertex has an odd degree, return 'no'. If 
all vertices have been checked having even degree, return 'yes'. The running time of this 
algorithm depends on the encoding of the graph. If G = (V, E) is encoded as a |V| x \V\ 
adjacency matrix with entries =number of edges connecting Vi and Vj, the the running 
time is C(|V| 2 ). Thanks to Euler, Eulerian Circuit is a tractable problem. The burghers 
of Konigsberg on the other hand had to learn from Euler, that they would never find a walk 
through their hometown crossing each of the seven bridges exactly once. 

2.1.2 Hamiltonian cycles 

Another decision problem is associated with the mathematician and Astronomer Royal of 
Ireland, Sir William Rowan Hamilton. In the year 1859, Hamilton put on the market a new 
puzzle called the Icosian game (Fig. ||). 

The generalization of the Icosian game calls for some definitions from graph theory: A 
closed walk in a graph is called a cycle if all its vertices (execpt the first and the last) are 
distinct. A Hamiltonian cycle is a cycle that contains all vertices of a graph. The generalization 
of the Icosian game then reads 
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Figure 4: Sir Hamilton's Icosian game: Find a route along the edges of of the dodecahedron 
(left), passing each corner exactly once and returning to the starting corner. A solu- 
tion is indicated (shaded edges) in the planar graph that is isomorphic to the dodec- 
ahedron (right). 

Hamiltonian Cycle: Given a graph G = (V, E). Does G contain a Hamilto- 
nian cycle? 

There is a certain similarity between Eulerian Circuit and Hamiltonian Cycle. In the 
former we must pass each edge once; in the latter, each vertex once. Despite this resemblance 
the two problems represent entirely different degrees of difficulty. The available mathemati- 
cal insights into Hamiltonian Cycle provide us neither with a polynomial algorithm nor 
with a proof that such an algorithm is impossible. Hamiltonian Cycle is intractable, and 
nobody knows why. 

2.1.3 Coloring 

Imagine we wish to arrange talks in a congress in such a way that no participant will be forced 
to miss a talk he would like to hear. Assuming a good supply of lecture rooms enabling us to 
hold as many parallel talks as we like, can we finish the programme within k time slots? This 
problem can be formulated in terms of graphs: Let G be a graph whose vertices are the talks 
and in which two talks are adjacent (joined by an edge) if and only if there is a participant 
whishing to attend both. Your task is to assign one of the k time slots to each vertex in such a 
way that adjacent vertices have different time slots. The common formulation of this problem 
uses colors instead of time slots: 
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/c-Coloring: Given a graph G = (V, E). Is there a coloring of the vertices of G 
using at most k different colors such that no two adjacent vertices have the same 
color? 

When k = 2 this problem is tractable — the construction of a polynomial algorithm is left as 
easy exercise. For k — 3 things change considerably: 3-COLORING is intractable. Note that 
for larger k the problem gets easier again: a planar graph is always colorable with 4 colors! 
This is the famous 4-color Theorem. 3-COLORING remains intractable even when restricted 
to planar graphs. 

2.1.4 Satisfiability 

We close this section with a decision problem that is not from graph theory but from Boolean 
logic. A Boolean variable x can take on the value (false) or 1 (true). Boolean variables can 
be combined in clauses using the Boolean operators 

- NOT 7 (negation): the clause x is true (x = 1) if and only if x is false (x = 0). 

- AND A (conjunction): the clause x\ A x 2 is true (x\ A x 2 — 1) if and only if both 
variables are true: x\ — 1 and x 2 — 1 

- OR V (disjunction): the clause x\ V x 2 is true (xi V x 2 — 1) if and only if at least one of 
the variables is true: x\ — 1 or x 2 — 1. 

A variable x or its negation If is called a literal. Different clauses can be combined to yield 
complex Boolean formulas, e.g. 

F 1 (x 1 ,x 2 ,x 3 ) = (xiVxo-Vxz) A (x 2 Vxj) A (xJV x 2 ) A (xJVx^). (4) 

A Boolean formula evaluates to either 1 or 0, depending on the assignment of the Boolean 
variables. In the example above Fi = 1 for x i = l,x 2 = l,x 3 = and Fi = for 
Xi — x 2 — x 3 — 1. A formula F is called satisfiable, if there is at least one assignment of the 
variables such that the formula is true. F± is satisfiable, 

F 2 (xi, x 2 ) = (xi V x 2 ) A12 Aii (5) 

is not satisfiable. 

Every Boolean formula can be written in conjunctive normal form (CNF) i.e. as a set of 
clauses C fe combined exclusively with the AND-operator 

F = C ± A C 2 A • • • A C m (6) 

where the literals in each clause are combined exclusively with the OR-operator. The exam- 
ples F 1 and F 2 are both written in CNF. Each clause can be considered as a constraint on the 
variables, and satisfying a formula means satisfying a set of (possibly conficting) constraints 
simultaneously. Hence 
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Satisfiability (SAT): Given disjunctive clauses Ci, C 2 , . . . , C m of literals, 
where a literal is a variable or negated variable from the set {x%, x 2 , ■ ■ ■ , x n }. Is 
there an assignment of variables that satisfies all clauses simultaneously? 

can be considered as prototype of a constraint satisfaction problem [|T5p. Fixing the number of 
literals in each clause leads to 

/c-SAT: Given disjunctive clauses C\, C 2 , ■ ■ . , C m of k literals each, where a 
literal is a variable or negated variable from the set {xi,x 2 , . . . ,x n }. Is there an 
assignment of variables that satisfies all clauses simultaneously? 

Polynomial algorithms are known for 1-SAT and 2-SAT QT7|] . No polynomial algorithm is 
known for general SAT and /c-SAT if k > 2. 



2.2 The classes V and MV 
2.2.1 Tractable problems 

Now we have seen enough examples to introduce two important complexity classes for deci- 
sion problems. 

The class of tractable decision problems is easy to define: it consists of those problems, 
for which a polynomial algorithm is known. The corresponding class is named V for "poly- 
nomial": 

Definition: A decision problem P is element of the class V if and only if it can 
be solved by a polynomial time algorithm. 

Eulerian Circuit, 2-Coloring, MST(D) etc. are in V. 



2.2.2 Nondeterministic algorithms 

The definition of the second complexity class involves the concept of a nondeterministic algo- 
rithm. A nondeterministic algorithm is like an ordinary algorithm, except that it may use one 
additional, very powerful instruction [fT8t] : 



goto both label 1 , label 2 

This instruction splits the computation into two parallel processes, one continuing from each 
of the instructions indicated by "label 1" and "label 2". By encountering more and more such 
instructions, the computation will branch like a tree into a number of parallel computations 
that potentially can grow as an exponential function of the time elapsed (see Fig. |5|). A non- 
deterministic algorithm can perform an exponential number of computations in polynomial 
time! In the world of conventional computers, nondeterministic algorithms are a theoretical 
concept only, but in quantum computing this may change (section [Q| ). We need the concept 
of nondeterminism to define the class MV of "nondeterministic polynomial" problems: 

Definition: A decision problem P is in the class J\fV, if and only if it can be 
solved in polynomial time by a nondeterministic algorithm. 



12 




Figure 5: Example of the execution history of a nondeterministic algorithm. 

Solubility by a nondeterministic algorithm means this: All branches of the computation will 
stop, returning either 'yes' or 'no'. We say that the overall algorithm returns 'yes', if any of 
its branches returns 'yes'. The answer is 'no', if none of the branches reports 'yes'. We say 
that a nondeterministic algorithm solves a decision problem in polynomial time, if the number 
of steps used by the first of the branches to report 'yes' is bounded by a polynomial in the size 
of the problem. 

We require polynomial solubility only for the 'yes '-instances of decision problem. This 
asymmetry between 'yes' and 'no' reflects the asymmetrie between the 'there is'- and 'for 
all'-quantifiers in decision problems: a graph G is a 'yes '-instance of Hamiltonian Cycle, 
if there is at least one Hamiltonian cycle in G. For a 'no '-instance, all cycles in G have to be 
non Hamiltonian. 

Note that the conventional (deterministic) algorithms are special cases of a nondetermin- 
istic algorithms (those nondeterministic algorithms that do not use the goto both instruction). 
It follows immediately that VCftfV. 

All decision problems we have discussed in the preceeding section are members of HV. 
Here's a nondeterministic polynomial algorithm for SAT: 
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Satisfiability(F) 

Input: Boolean formula F(xi, . . . , x n ) 

Output: 'yes' if F is satisfiable, 'no' otherwise 

begin 

for % = 1 to n 

goto both label 1 , label 2 
label I: Xi = true; continue 
label 2: Xj = false; continue 
end 

if F(x\, . . . , x n ) = true then return 'yes' 

else return 'no' 

end 



The for-loop branches at each iteration: in one branch x,- L = true, in the other branch 
Xi = false (the continue instruction starts the next iteration of the loop). After executing the 
for-loop we have 2" branches of computation, one for each of the possible assignments of n 
Boolean variables. 

The power of nondeterministic algorithms is that they allow the exhaustive enumeration of 
an exponentially large number of candidate solutions in polynomial time. If the evaluation of 
each candidate solution (calculating F(xi, . . . , x n ) in the above example) in turn can be done 
in polynomial time, the total nondeterministic algorithm is polynomial. For a problem from 
the class J\fV, the sole source of intractability is the exponential size of the search space. 



2.2.3 Succinct certificates 

There is a second, equivalent definition of J\fV, based on the notion of a succinct certificate. 
A certificate is a proof. If you claim that a graph G has a Hamiltonian cycle, you can proof 
your claim by providing a Hamiltonian cycle. Certificates for Eulerian Circuit and k- 
COLORING are an Eulerian circuit and a valid coloring. A certificate is succinct, if its size is 
bounded by a polynomial in the size of the problem. The second definition then reads 

Definition: A decision problem P is element of the class NT if and only if for 
every 'yes '-instance of P there exists a succinct certificate that can be verified in 
polynomial time. 



The equivalence of both definitions can easily be shown [ITS]. The idea is that a succinct 



certificate can be used to deterministically select the branch in a nondeterministic algorithm 
that leads to a 'yes '-output. 

The definition based on nondeterministic algorithms reveals the key feature of the class 
NT more clearly, but the second definition is more usefull for proving that a decision problem 
is in NT. As an example consider 

COMPOSITENESS: Given a positive integer N. Are there integer numbers p > 1 
and q > 1 such that N = pql 
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A certificate of a 'yes' instance N of Compositeness is a factorization N = pq. It is 
succinct, because the number of bits in p and q is less or equal the number of bits in N, and it 
can be verified in quadratic time by multiplication. Hence CompositenessgA/'7 : '. 

Most decision problems ask for the existence of an object with a given property, like a 
cycle which is Hamiltonian or a factorization with integer factors. In these cases, the desired 
object may serve as a succinct certificate. For some problems this does not work, however, 
like for 

Primality: Given a positive integer N. Is iV prime? 

Primality is the negation or complement of Compositeness: the 'yes'-instances of the 
former are the 'no '-instances of the latter and vice versa. A succinct certificate for Primality 
is by no means obvious. In fact, for many decision problems in NT no succinct certificate 
is kown for the complement, i.e. it is not known whether the complement is also in MV . For 
Primality however, one can construct a succinct certificate based on Fermat's Theorem flTjp. 
Hence PrimalitygA/'P. 



2.2.4 A first map of complexity 



Hamiltonian-Cycle 



SAT 



3-SAT 

Graph-Isomorphism 



NP 



TSP(d) 

Primality 
3-Coloring 

Compositeness 



Eulerian-Circuit 


2-Coloring 


2-SAT 






Assignment(d) 


MST(d) 





Figure 6: A first map of complexity. All problems indicated are defined within the text. Prob- 
lems with a (D) are decision variants of optimization problems. 

Fig. ^| summarizes what we have achieved so far. The class MV consists of all decision 
problems whose sole source of difficulty is the size of the search space which grows exponen- 
tially with the size of the problem. These problems are intractable unless some mathematical 
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insight provides us with a polynomial shortcut to avoid exhaustive search. Such an insight 
promotes a problem into the class V of polynomially soluble problems. 

The class MV not only contains a large number of problems with important applications, 
but additionally represents a real challenge: all problems in MV still have a chance to be in 
V. A proof of non-existence of a polynomial algorithm for a single problem from MV would 
establish that V ^ MV . As long as such a proof is missing, 

V = MV (7) 
represents the most famous open conjecture in theoretical computer science. 

2.3 ./VP-completeness 
2.3.1 Polynomial reductions 

The computational complexities of two problems Pi and P 2 can be related to each other by 
constructing an algorithm for Pi that uses an algorithm for P 2 as a "subroutine". Consider the 
following algorithm that relates Hamiltonian Cycle to TSP(D): 

Hamiltonian Cycle(C7) 
Input: Graph G = (V, E) 

Output: 'yes' if G contains a Hamiltonian cycle, 'no' otherwise 

(1) begin 

(2) n := \V\ 

(3) for i — 1 to n 

(4) forj = lton 

(5) if (vi, vj) e E then d^ : = 1 

(6) else dij :— 2 

(7) if TSP-decision (d, B := n) = 'yes' then return 'yes' 

(8) else return 'no' 

(9) end 



This algorithm solves Hamiltonian Cycle by solving an appropriate instance of TSP(D). 
In the for-loops (lines 3-5) a distance matrix d is set up with entries d^ = 1 if there is an edge 
(vi, Vj) in G and d^ = 2 otherwise. Now a Hamiltonian cycle in G is valid tour in the corre- 
sponding TSP with all intercity distances having length I, i.e. with total length n. Conversely, 
suppose that the TSP has a tour of length n. Since the intercity distances are either 1 or 2 
and a tour sums up n such distances, a total length n implies that each pair of successively 
visited cities must have distance 1, i.e. the tour follows existing edges in G and corresponds 
to a Hamiltonian cycle. Hence the call to a subroutine that solves TSP(D) (line 7) yields a 
solution to Hamiltonian Cycle. 

How does this algorithm relate the computational complexity of HAMILTONIAN CYCLE 
to that of TSP(D)? Note that this is a polynomial algorithm if the call to the TSP(D)-solver 
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is considered as elementary operation. If someone comes up with a polynomial algorithm for 
TSP(D), we will instantly have a polynomial algorithm for Hamiltonian Cycle! We say 
that Hamiltonian Cycle is polynomially reducible to TSP(D) and write 

Hamiltonian Cycle < TSP(D). (8) 

In many books, polynomial reducibility is denoted by 'oc' instead of '<'. We follow [ |20"| ] and 
use '<' because this notation stresses an important consequence of polynomial reducibility: 
the existence of a polynomial reduction from P\ to P 2 excludes the possibility that P 2 can be 
solved in polynomial time, but P x cannot. Hence P x < P 2 can be read as P x cannot be harder 
than P 2 . Here is the (informal) definition: 

Definition: We say a problem P x is polynomially reducible to a problem P 2 and 
write Pi < P 2 if there exists a polynomial algorithm for P 1 provided there is a 
polynomial algorithm for P 2 . 

2.3.2 A/"P-complete problems 

Here are some other polynomial reductions that can be verified similar to Eq. |8|: 



See [0,^1]] for the corresponding reduction algorithms. Polynomial reducibility is transitive: 
Pi < P 2 andP 2 < P 3 imply Pi < P 3 . From transitivity and Eqs. |]and^|it follows that each of 
SAT, 3-SAT, 3-Coloring and Hamiltonian Cycle reduces to TSP(D), i.e. a polynomial 
algorithm for TSP(D) implies a polynomial algorithm for all these problems ! This is amazing, 
but only the beginning. The true scope of polynomial reducibility was revealed by Stephen 



Cook in 1971 [ |22| ] who proved the following, remarkable theorem: 

Theorem: (Cook, 1 97 1 ) All problems in MV are polynomially reducible to S AT , 

VP E MV : P < SAT (10) 



This theorem means that 



SAT < 
3-SAT < 
3-COLORING < 



3-SAT 

3-coloring 
Hamiltonian Cycle 



(9) 



1 . no problem in MV is harder than SAT or SAT is among the hardest problems in MV . 

2. a polynomial algorithm for SAT would imply a polynomial algorithm for every problem 
in MV, i.e. would imply V = MV. 

It seems as if SAT is very special, but according to transitivity and Eqs. |9| and |8] it can be 
replaced by 3-SAT, 3-COLORING, HAMILTONIAN Cycle or TSP(D). These problems form 
a new complexity class: 
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Definition: A problem P is called AfP-complete if P £ NT and Q < P for all 

Q £ NT 

The class of A^P-complete problems collects the hardest problems in NT. If any of them 
has an efficient algorithm, then every problem in NT can be solved efficiently, i.e. T = NT. 
This is extremely unlikely, however, considered the futile efforts of many brilliant people to 
find polynomial algorithms for problems like Hamiltonian Cycle or TSP(D). 



Hamiltonian-Cycle 



SAT 




Graph-Isomorphism 


Primality 
Compositeness 


Eulerian-Circuit 


2-Coloring 


2-SAT 




MST(d) 


Assignment(d) 



Figure 7: The map of complexity revisited. 



2.3.3 The map of NT 

Since Cook's Theorem many problems have been shown to be A^P-complete. A compre- 
hensive, up-to-date list of hundreds of A/'P-complete problems can be found in the web [E3p . 



Our map of NT needs some redesign (Fig. |7|). It turns out that all the intractable problems we 
have discussed so far are jVP-complete - except Compositeness and Primality. For both 
problems neither a polynomial algorithm is known nor a polynomial reduction that clasifies 
them J\fV -complete. Another NT problem which resists classification in either T or NT is 
this: 

Graph Isomorphism: Given two graphs G = (V, E) and C7(V, E) on the same 
set of nodes. Are G and G isomorphic, i.e. is there a permutation n of V such that 
G = Tt(G), where by 7r(C7) we denote the graph (V, {[7t(m), tt(v)] : [u, v] £ E})1 

There are more problems in NT that resists a classification in T or NT, but none of these 
problems has been proven not to belong to T or NT. What has been proven is 
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(a) 




Figure 8: Three tentative maps of MV. (b) can be ruled out. It is very likely (but not sure) 
that (a) is the correct map. 

Theorem: If V ^ MV, then there exist MV problems which are neither in V nor 
are they A/'P-complete. 

This Theorem [ |24| ] rules out one of three tentative maps of MV (Fig. ||). 

2.4 Beyond MV 

2.4.1 Optimization problems 

How does the classification of decision problems relates to optimization problems? The gen- 
eral instance of an optimization problem is a pair (F, c), where F is the set of feasible solutions 
and c is a cost function c : F — > 5L We will consider only combinatorial optimization where 
the set F is countable. A combinatorial optimization problem P comes in three different 
flavors: 

1. The optimization problem P(0): Find the feasible solution f* E F that minimizes the 
cost function. 

2. The evaluation problem P(E): Find the cost c* = c(f*) of the minimum solution. 

3. The decision problem P{D): Given a bound B e R, is there a feasible solution / 6 F 
such that c(f) < Bl 

Under the assumption that the cost function c can be evaluated in polynomial time, it is 
straightforward to write down polynomial reductions that establish 

P(D) <P(E) <P(0). (11) 

If the decision variant of an optimization problem is A^P-complete, there is no efficient algo- 
rithm for the optimization problem at all — unless V = NT . An optimization problem like 
TSP, whose decision variant is A^P-complete is denoted A/'P-hard. 

Does a polynomial algorithm for a decison problem imply a polynomial algorithm for the 
optimization or evaluation variant? For that we need to proof the reversal of Eq. [IT], 

P(0) < P(E) < P(D). (12) 
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P(E) < P(D) can be shown to hold if the cost of the optimum solution is an integer with 
logarithm bounded by a polynomial in the size of the input. The corresponding polynomial 
reduction evaluates the optimal cost c* by asking the question "Is c* < BT for a sequence of 
values B that approaches c*, similar to the bisection method to find the zeroes of a function. 

There is no general method to prove P(0) < P(E), but a strategy that often works can 
be demonstrated for the TSP: Let c* be the known solution of TSP(E). Replace an arbitrary 
entry dij of the distance matrix with a value c > c* and solve TSP(E) with this modified 
distance matrix. If the length of the optimum tour is not affected by this modification, the link 
ij does not belong to the optimal tour. Repeating this procedure for different links one can 
reconstruct the optimum tour with a polynomial number of calls to a TSP(E)-solver, hence 
TSP(O) < TSP(E). 

2.4.2 Counting problems 

So far we have studied two related styles of problems: Decision problems ask whether a 
desired solution exists, optimization problems require that a solution be produced. A third 
important and fundamentally different kind of problem asks, how many solutions exist. The 
counting variant of SAT reads 

#S AT: Given a Boolean expression, compute the number of different truth assign- 
ments that satisfy it 

Similarly, #Hamiltonian Cycle asks for the number of different Hamiltonian cycles in a 
given graph, #TSP for the number of different tours with length < B and so on. 

Definition: A counting problem #P is a pair (F, d), where F is the set of all 
feasible solutions and d is a decision function d : F i— > {0, 1}. The output of #P 
is the number of / E F with d(f) = 1. The class #V (pronounced "number P") 
consists of all counting problems associated with a decision function d that can be 
evaluated in polynomial time. 

Like the class MV, #V collects all problems whose sole source of intractability is the number 
of feasible solutions. A polynomial algorithm for a counting problem #P implies a polynomial 
algorithm for the associated decision problem P: P < #P. Hence it is very unlikely that 
#SAT can be solved efficiently. In fact one can define polynomial reducibility for counting 
problems and prove that all problems in #V reduce polynomially to #SAT @: 

Theorem: #SAT is #P-complete. 

As you might have guessed, #Hamiltonian Cycle and #TSP are ttP-complete, too. De- 
spite the similarity between MV and #V, counting problems are inherently harder than de- 
cision problems. This is documented by those #P-complete problems, for which the corre- 
sponding decision problem can be solved in polynomial time, the classical example being the 
problem of calculating the permanent of a matrix [j25|]. 
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3 Computational complexity and physics 



3.1 Quantum parallelism 



In a seminal paper pq], Richard Feynman pointed out that a system of n quantum particles is 
exponentially hard to simulate on a classical computer. The idea of quantum computing is to 
reverse this situation and simulate a classically hard (i.e. exponential!) problem in polynomial 
time on a computer made of quantum devices. 

A quantum computer processes qubits, quantum two-state systems |0), 1 1)). A key feature 
of a quantum computer is that its registers can hold and process linear superpositions of all 2 n 
product states of n qubits like 

: 1 

7= Yl 1^2 ••-^). (13) 



2 n 

ii,i2,...,i„=0 

Using this feature it is not very difficult to construct a quantum computer capable of computing 
any function f(x\, . . . ,x n ) of n Boolean variables simultaneously for all 2 n possible input 
values — in theory at least. This quantum parallelism resembles very much a nondeterministic 
algorithm with its goto both instruction and its exponentially branching execution tree. Is 
quantum parallelism the key to exponential computing power? The problem is how to extract 
the exponential information out of a quantum computer. When we defined nondeterministic 
solubility we did not care about how to "spot" the single 'yes'-answer among the 2 n 'no'- 
answers. This works fine for a theoretical concept, but for a practical computer reading the 
output matters a lot. 

In order to gain advantage of exponential parallelism, it needs to be combined with another 
quantum feature known as interference. The goal is to arrange the computation such that 
constructive interference amplifies the result we are after and destructive interference cancels 
the rest. Due to the importance of interference phenomena it is not surprising that calculating 
the Fourier transform was the first problem that undergoes an exponential speedup: from 
C(nlogn) on a classical to 0(\og 2 n) on a quantum computer. This speedup was the seed for 
the most important quantum algorithm known today: Shor's algorithm to factor an integer in 
polynomial time [ETJl . 



Although Shor's algorithm has some consequences for public key cryptography, it does not 
shatter the world of NT: remember that Compositeness is in HV, but not AfP-complete. 
Hence the holy grail of quantum computing, a polynomial time quantum algorithm for an 
jVP-complete problem, is yet to be discovered! 

See [ |28| ] or www.qubit.org to learn more. 



3.2 Analytical solubility of Ising models 

Some problems in statistical physics have been exactly solved, but the majority of problems 
has withstand the efforts of generations of mathematicians and physicists. Why are some 
problems analytically solvable whereas others, often similar, are not? Relating this question 
to the algorithmic complexity of evaluating the partition function gives us no final answer 
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but helps to clarify the borderline that separates the analytically tractable from the intractable 
problems. 




Figure 9: Onedimensional Ising spin glass with periodic boundary conditions. The partition 
sum of this system can be calculated in polynomial time. 



As an example consider the Ising spin glass on a general graph G [E5p . Let a = (<7i, . . . , 



be an assignment of Ising spins Oi = ±1 ("up" or "down"). The energy of a configuration a is 

E ( a ) = -J2 3 ^ 0i °3 - H J2 a " (14) 

(ij) » 

where H is the external magnetic field, Jjj are the coupling constants and the first summation 
is over all edges in G. The fundamental problem in statistical mechanics is to determine the 
partition function 

Z N (G) =J2 e ~ ma) - ( 15 ) 

Evaluating the sum directly requires 0(2 N ) operations. The notion "analytic solution" has no 
precise definition, but as a minimum requirement we want this number to be reduced from 
being exponential to being polynomial in N. As an example consider the well known transfer 
matrix solution of the one dimensional Ising glass with periodic boundary conditions and 
coupling J k between spins a k and a k+ \ (Fig. ||), 

JL ( e P{.h+H) e -/3J k \ 

^(ring) = Trj{ ^ ^ ^ {Jk _ H) j, (16) 

which can be evaluated in O(N) elementary operations. Since any exact solution must include 
all numbers J k , this is the best we can expect. In the homogeneous case J k = J, where we 
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can calculate the product of transfer matrices, 



A+ + A^ with A± = e pj 



cosh((3H) ± Jcosh 2 (f3H) - 2e~ 2 ' 3H sinh(2/3#) 



(17) 



the evaluation complexity drops to O(logiV) using fast exponentiation. 
Writing the partition sum as 

Z{G) = Y J <E k )e-^ (18) 

where the sum is over all possible energy values, it becomes obvious that calculating Z is 
closely related to the ttPproblem of determining n(Ek). For general finite graphs G, this 
problem has proven to be ttP-complete [|3T)|,|3~T]], so there is no hope to find an analytical 
solution (even in the weak sense above). This situation hardly improves if we restrict the 
graphs to the more "physical" crystal lattices: computing the partition function for a finite 
sublattice of a non-planar crystal lattice is ttP-complete [02p. This includes every crystal 



lattice in d > 2, the d=2 model with next-nearest neighbor interactions, two coupled d = 2 
models etc.. It also includes all models with d > 2 and external field H, since these can 
be transformed into zero-field models on an augmented graph G which is non-planar unless 
the underlying lattice graph G is onedimensional. The construction of G from G is easy, 
just adjoin an additional vertex (spin) cr to G and let the additional edges a ai have constant 
interaction energy J i = H. The partition function of the augmented system reads 

Z(G) = ^e+EW^E^^E^ (19) 

where the additional factor comes from the new spin cr = ±1. From this expression it is easy 
to see that Z(G) equals two times the partition function Z(G) in field H. 

But where are the soluble Ising models? It has been proven that the partition function 
of Ising systems on planar crystal lattices can be evaluated in polynomial time [031]. This 
includes the celebrated Onsager solution of the square ferromagnet J34[ | as well as the oned- 
imensional example from above. It turns out that the Ising model's partition sum can be 
calculated in polynomial time for all graphs of fixed genus g [05], E^p . A graph has genus g if 



it can be drawn on an orientable surface of genus g (a sphere with g "handles" attached to it) 
without crossing the edges. Planar graphs have genus 0, toroidal graphs have genus 1 and so 
on. For the crystal lattices in d > 2, the genus increases monotonically with the number of 
spins, i.e. it is not fixed [05j]. 

The mechanism for proving tractability or intractability is the same in statistical mechanics 
as in computational complexity: polynomial reduction. Barahona [fS3J[ for example applies 
a reduction of the AAP-hard problem Max Cut to the Ising spin glass in 3d to proof the 
TVP-hardness of the latter. A reduction of the planar Ising model to Minimum Weight 
Matching on the other hand proofs the tractability of the former, since Minimum Weight 
Matching can be solved in polynomial time. 

Note that in our classification of spin systems the nature of the couplings is not significant. 
A frustrated, glassy system with random couplings Jjj of both signs is in the same class as 
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the homogeneous ferromagnet with Jjj = J > as long as the underlying graph G is the 
same. In our onedimensional example we did not discriminate these cases: they are both 
polynomial. This situation changes of course if we consider the ground states rather than the 
complete partition function. Here the nature of the couplings matters a lot: finding the ground 
states in pure ferromagnetic systems is trivial on all lattices, whereas it is J\fV-hard for glassy 
systems with positive and negative couplings on non-planar lattices []33[], 

A lot of other problems arising in statistical physics can be classified according to the 
computational complexity to evaluate their partition function, see J37| ] and references therein. 
It turns out that all the problems known to have an exact solution [ JT5[ ] can be evaluated in 
polynomial time. Problems that are ttP-complete, however, are very unlikely to be exactly 
solvable. Anyone looking for an exact solution of such a problem should keep in mind, that he 
or she is simultaneously attacking TSP, Hamiltonian cycle, SAT and all the other NP- 
hard problems. In statistical mechanics the focus is on results for the thermodynamic limit 
iV — > oo rather than for finite systems, however. It is not clear how much of the "hardness" 
survives in this limit. 



3.3 Probabilistic analysis of combinatorial problems 

Problems from combinatorial optimization can formally considered as models in statistical 
mechanics. The cost function is renamed Hamiltonian, random instances are samples of 
quenched disorder, and the ground states of the formal model correspond to the solutions 
of the optimization problems. In this way methods developed in the framework of statisti- 
cal mechanics of disordered systems become powerful tools for the probabilistic analysis of 
combinatorial problems [ |39| ] . 

Statistical mechanics methods have been applied for example to the TSP [ [401 , [41] ], GRAPH 



Partitioning [^2J and &;-SAT [|43|,^4|]. A particular nice example of this approach is the 
analysis of Assignment (also called Bipartite Matching): Given an iV x N matrix with 
non-negative entries a it j > 0. Find 

JV 

E* N = mm ^ a iMi) ( 2 °) 

i=l 

where the minimum is taken over all permutations a of (1, . . . , N). 

A probabilistic analysis aims at calculating average properties for an ensemble of random 
instances, the canonical ensemble being random numbers a^j drawn independently from a 
common probability density p(a). Using the replica method from statistical physics, Marc 
Mezard and Giorgio Parisi p3| ] found (among other things) 

lim (E* N ) = ^, (21) 

N—kx 



where (•) denotes averaging over the a^j. A rigorous proof of eq. ET] has been presented 



recently [ |46| ] and represents one of the rare cases where a replica result has been confirmed by 
rigorous methods. 
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Some years after the replica-solution, Parisi recognized that for exponentially distributed 
matrix elements ( p(a) = e~ a ) the average optimum for N — 1 and N = 2 is 



(El) = 1 (E*) = 1 + 



2 2 ' 



(22) 



From this and the fact that the replica result for N — > oo can be written as 

2 oo 1 
7T x - l 



k=l 



(23) 



he conjectured []37|] that the average optimum for finite systems is 



N 1 



(24) 



fc=i 



Parisi's conjecture is supported by numerical simulations, but no formal proof has been found 
despite some efforts 

Note that Eqs. [22| and only hold for p(a) = e~ a , whereas eq. |T]is valid for all densities 
with p(a — > 0) = 1. For the uniform density on [0, 1] the first terms are 



(25) 



If you can you guess the expression for general, finite N in this case, please send me an email. 

Sometimes a statistical mechanics analysis not only yields exact analytical results but also 
reveals features that are important to design and understand algorithms. A recent example is 
the analysis of the Davis-Putnam- Algorithm for SAT []50|,|5H]. Another example is given by 
the number partitioning problem NPP fl52"[], an A^P-hard optimization problem, where it has 



been shown, that for this particular problem no heuristic approach can be better than stupid 
random search [|53|-|55|] . 



3.4 Phase transitions in computational complexity 

The theory of computational complexity is based entirely on worst-case analysis. It may 
happen that an algorithm requires exponential time on pathological instances only. A famous 
example is the Simplex Method for Linear Programming. Despite its exponential worst- 
case complexity it is used in many applications to solve really large problems. Apparently the 
instances that trigger exponential running time do not appear under practical conditions. 

Now Linear Programming is in V thanks to the Ellipsoid algorithm [gp, but similar 
scenarios are observed for A/'P-hard problems. Take 3-SAT as an example. Generating ran- 
dom instances with variables and M clauses and feeding them to a smart but exhaustive 
algorithm one observes polynomial running time unless the ratio M/N is carefully adjusted. 
If M/N is too low, the problem is underconstrained, i.e. has many satisfying solutions, and a 
clever algorithm will find one of these quickly. If M/N is too large, the problem is overcon- 
strained, i.e. has a large number of contradictory constraints which again a clever algorithm 
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will discover quickly [p6Q . The real hard instances are generated for ratios a = M/N close to 
a critical value a c [ j57[ ] . 

The transition from underconstrained to overconstrained formulas in in 3-SAT bears the 
hallmarks of a phase transition in physical systems. The control parameter is a, the order 
parameter is the probability of the formula being satisfiable. Similar phase transitions have 
been discovered and analyzed with methods from statistical mechanics in other computational 
problems. See the special issue of Theoretical Computer Science on "Phase Transitions in 
Combinatorial Problems" for a recent overview of this interdisciplinary field [|58|]. 
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